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DETAILED ACTION 



1. This action is responsive to communications: IDS's received8/27/2004, and 
Amendments filed 1/27/2004 and 8/27/2004. 

2. Claims 1-34 are pending in the case. Claims 1, 7, 10, 13, 20, 25 and 29 are independent 
claims. 



3. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

4, Claims 10-12 and 25-27 remain rejected under 35 U.S.C- 101 because the claimed 
invention is directed to non-functional descriptive data stored on a medium. 

The claims are not directed to statutory subject matter because the claimed subject 
matter: Does not fall within one of the four statutory classes of inventions under 101; and/or 
falls, by analogy, within the printed matter exception 101. 

Data structures do not fall within one of the four statutory classes of invention under 
101: process, machine, manufacture, and composition of matter. A data structure is clearly 
neither a "process" nor a "machine." With regard to the other statutory classes, the Supreme 
Court in Diamond v. Chakrabartv . 206 USPQ 193 (S. Ct. 1980), has defined a "manufacture" as 
"the production of articles for use from raw materials prepared by giving to these materials new 
forms, qualities, properties, or combinations whether by hand labor or by machinery" and has 
defined a "composition of matter" as "all compositions of two or more substances and ... all 
composite articles, whether they be results of chemical union, or of mechanical mixture, or 
whether they be gases, fluids, powders, or solids." Id at 195-196. Clearly, a data structure, per 



Claim Rejections - 35 USC §101 
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se, cannot be considered a "manufacture" since a data structure is not produced from raw 
materials and has no tangible, physical form or structure. Likewise, a data structure cannot be 
considered a "composition of matter" since a data structure is not a composition of substances or 
composite articles as contemplated by the Supreme Court. Accordingly, since a data structure 
does not fall within one of the four statutory classes of inventions under 101, the claims are not 
directed toward statutory subject matter. 

Lastly, a data structure is considered non-statutory subject matter by analogy to the 
"printed matter" exception under 101. See In re Miller , 164 USPO 46. 49 fCCPA 1969). Like 
printed matter, a data structure, in and of itself, is merely an arrangement of data and nothing 
more. Furthermore, claims drawn to printed matter may be non-statutory even though the claims 
recite the structure on which the printed matter is printed: 

The mere arrangement of printed matter on a sheet or sheets of paper, in book 
form or otherwise, does not constitute "any new and useful art, machine, 
manufacture, or composition of matter," or "any new and useful improvements 
thereof," as provided in section 4886, of the Revised Statutes [the predecessor to 
35U.S.C, 101]. 

(emphasis in original). In re Russell 9 USPQ 181, 182 (CCPA 1931). At best, the claims as a 
whole describe a data structure stored in a computer system. Accordingly, like printed matter 
"stored" on a sheet of paper, a data structure stored in a computer system fails to present 
statutory subject matter. 
MPEP § 2106 states: 

When nonfunctional descriptive material is recorded on some computer-readable medium, it 
is not statutory since no requisite functionality is present to satisfy the practical application 
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requirement. Merely claiming nonfunctional descriptive material stored in a computer- 
readable medium does not make it statutory. Such a result would exalt form over substance. 
In re Sarkar, 588 F.2d 1330, 1333, 200 USPQ 132, 137 (CCPA 1978) ( M [E]ach invention 
must be evaluated as claimed; yet semantogenic considerations preclude a determination 
based solely on words appearing in the claims. In the final analysis under 101, the claimed 
invention, as a whole, must be evaluated for what it is.") (quoted with approval in Abele, 684 
F.2d at 907, 214 USPQ at 687). See also In re Johnson, 589 F,2d 1070, 1077, 200 USPQ 
199, 206 (CCPA 1978) ("form of the claim is often an exercise in drafting"). 



Although the independent claim recites "data structure" in the preamble, the body of the 
claim merely recites a collection of fields. Use of the broad phrase "correlated through" is 
insufficient to establish the necessary functional interrelationship between elements to constitute 
a true data structure. As such, the claims recite non- functional descriptive data stored on a 
medium. The claims specifically recite two descriptive fields, which describe data. Merely 
encoding a non- functional descriptive data on a statutory medium does not make the claim 
statutory. See MPEP 2106 F/.B. 1 for a complete discussion including legal basis and rationale. 



Claim Rejections - 35 USC §103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 1-12, 20, 22, 23 and 25-34 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Synchronized Multimedia Integration Language (SMIL) 1.0 
Specification," W3C Recommendation 15-June-1998, herein referred to as W3C, in view of 



deVries et aL (USPN 6332144— (priority date 3/11/1998). 
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Regarding independent claim 1, W3C discloses a system (p.26, ex. 4) for associating 
annotations, contained in the <text . . . /> tag, to the plurality of media streams, representing 
different versions of multimedia content, that would be contained in ellipses found at lines 6, 9, 
and 12 of the example, as explained in the header of the example. The text is contained with a 
<par> tag (§ 4.2. 1), with the <switch> block. Within the switch block, as noticed in the 
examples text, are "alternative parts designed for screens with different resolutions, and 
bit-depths." The ellipses at 6, 9, and 12, are clearly meant to be the alternative parts. These are 
the different versions of multimedia content. They would be represented by one of the tags 
disclosed in §4.2.3. Both those tags, and <text> when contained within a <par> can be 
associated to particular time frames (p. 12). As the text is assigned to a particular "temporal 
location in the multimedia presentation," it is equivalent to an annotation as defined in the 
specification, (page 2, line 19 - page 3, line 7), even meeting the specific example of "temporal 
location ." 

While there is no explicit mention of multimedia servers, W3C does however disclose that both 
the annotations and media are referenced by a URI (p. 19, "src"), which were well-known to 
refer to servers. deVries et al. (deVries) discloses that media and annotations can be stored on 
servers (FIG IB, 22 and 26), the annotation server inherently having a database of annotations. 
Therefore it would have been obvious to one of ordinary skill in the art at the time of the 
invention to combine deVries and W3C in order to effectively store the annotations (col. 14, 11. 
5-8), media (col. 4, 11. 34-08), and relations between the two (FIG. 7). 
Regarding independent claim 29, W3C discloses a system (p.26, ex. 4) for associating 
annotations, contained in the <text . . . /> tag, to the plurality of media streams, representing 
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different versions of multimedia content, that would be contained in ellipses found at lines 6, 9, 
and 12 of the example, as explained in the header of the example. The text is contained with a 
<par> tag (§ 4.2. 1), with the <switch> block. Within the switch block, as noticed in the 
example's text, are "alternative parts designed for screens with different resolutions, and 
bit-depths." The ellipses at 6, 9, and 12, are clearly meant to be the alternative parts. These are 
the different versions of multimedia content. They would be represented by one of the tags 
disclosed in §4.2.3. Both those tags, and <text> when contained within a <par> can be 
associated to particular time frames (p. 12). As the text is assigned to a particular "temporal 
location in the multimedia presentation," it is equivalent to an annotation as defined in the 
specification, (page 2, line 19 - page 3, line 7), even meeting the specific example of "temporal 
location." 

While there is no explicit mention of multimedia servers, W3C does however disclose that both 
the annotations and media are referenced by a URI (p. 19, "src"), which were well-known to 
refer to servers. deVries et al. (DeVries) discloses that media and annotations can be stored on 
servers (FIG IB, 22 and 26), the annotation server inherently having a database of annotations. 
DeVries teaches that the server is configured to provide on of the media stream to a requester 
(col. 7, 11. 26-40). It would have been obvious to one of ordinary skill in the art at the time of the 
invention to combine deVries and W3C in order to effectively store the annotations (col. 14, 11. 
5-8), media (col. 4, 11. 34-08), and relations between the two (FIG. 7). 
Regarding dependent claims 2 and 30, deVries discloses the streams can be audio or video 
(col. 1,11. 59-61). 
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Regarding dependent claim 3 and 31, W3C discloses different time compressed versions (p. 
26, ex. 2). 

Regarding dependent claim 4 and 32, W3C discloses versions having different resolutions (p. 
26, ex. 4). 

Regarding dependent claim 5 and 33, W3C discloses an identifier for each of the plurality of 
media streams (p. 19, "src"). 

Regarding dependent claim 6 and 34, W3C discloses the identifier is a URI (p. 19, "src") 
equivalent to a URL. 

Regarding independent claim 7, W3C discloses an SMIL document that correlates an 
annotation with different versions of the same multimedia content as described in the rejection of 
claim 1 . It is the position of the Office that a data structure is equivalent to an XML document 
such as a SMIL. W3C does not disclose storing the annotation. deVries discloses a storage 
device to store the annotation (FIG. IB, 26). It would have been obvious to combine deVries 
and W3C to store the annotation so that it is correlated through the data structure by storing the 
annotation in the storage device at the location referenced by the URI in the <text/> tag. 
Regarding dependent claim 8, W3C discloses a version list identifying multiple different 
versions (p. 26, ex. 4). 

Regarding independent claims 10, 25 and 28, W3C teaches a data field representing an 
annotation corresponding to multimedia content, and a data field correlated so that the field 
identifies a plurality of different versions of the multimedia content to which the annotation 
corresponds (p. 26, ex. 4). 
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Regarding dependent claims 11 and 26, W3C teaches that identifiers are intended to be 
included at the ellipses in example 4, on page 26. 

Regarding dependent claim 9, 12, and 27, W3C teaches that URLs, equivalent to URL's are 
intended to be included at the ellipses in example 4, on page 26. 

Regarding independent claim 20 and 23, W3C discloses receiving an indication of the version 
of media content (p. 18, 11 1-3). W3C also discloses that annotations are corresponding to a 
plurality of different versions of the media content (p. 26, ex. 4). W3C is silent on providing the 
annotations to he the user. DeVries discloses providing presenting media and annotations to a 
user (col. 19, 11. 29-42). It would have been obvious to one of ordinary skill in the art at the time 
of the invention to combine W3C and deVries as the markup language of W3C was intended to 
describe how media should interact with each other when it is displayed. 
Regarding dependent claim 22, while neither deVries nor W3C explicitly teach comparing 
identifiers, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to compare the identifiers to find out what media the version was a part of. 
7. Claims 13-15, 18-19, and remain rejected under 35 U.S.C 103(a) as being 
unpatentable over W3C in view of Logan et al. (USPN 5735216-^atented 3/24/1998). 
Regarding independent claim 13 and 19, W3C teaches associating an annotation with a set of 
media streams, as set forth in the rejection of claim 1 above, however, is silent as to the creation 
of the annotations. Logan teaches receiving a user request to create a new annotation (col. 32, 11. 
53-56). It would have been obvious to one of ordinary skill in the art at the time of the invention 
to include a method of creating a new annotation, as it would be allow easier addition of 
annotations. 
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Regarding dependent claim 14, W3C does not explicitly disclose associating a new record with 
a previously generated record. W3C does disclose annotations in <par> and <seq> tags (§§ 
4.2. 1 -.4. 1.2). Upon the creation of new annotation as recited in claim 13, it would have been 
obvious to one of ordinary skill in the art at the time of the invention to associate the annotation 
within the tags to allow the annotation to be synched with the media stream. 
Regarding dependent claim 15, it would have been inherent to identify the streams before 
associating anything with them. 

Regarding dependent claim 18, W3C discloses a record of the set of media streams (p. 26, 
ex.4). It would have been obvious to one of ordinary skill in the art at the time of the invention 
to generate and store this record so that it can be read later. 

8. Claims 16-17 and 24 remain rejected under 35 U.S.C. 103(a) as being unpatentable 
over W3C and Logan as applied to claim 13 above, and further in view of deVries. 
Regarding dependent claim 16, while there is no explicit mention of multimedia servers in 
W3C and Logan, W3C does however disclose that both the annotations and media are referenced 
by a URI (p. 19, "src"), which were well-known to refer to servers. deVries et al. (DeVries) 
discloses that media and annotations can be stored on servers (FIG IB, 22 and 26), the 
annotation server inherently having a database of annotations. Therefore it would have been 
obvious to one of ordinary skill in the art at the time of the invention to combine deVries and 
W3C in order to effectively store the annotations (col. 14, 11. 5-8), media (col. 4, 11, 34-08), and 
relations between the two (FIG. 7). 

As the annotations would have been stored on the server, it would have been obvious to 
communicate with server and identify the streams before associating anything with them. 
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Regarding dependent claim 17, W3C discloses a set of identifiers, each identifier uniquely 
identifying one of the sets of streams (p. 26, ex. 4). 

Regarding dependent claim 24, W3C is silent on providing annotations to a user. DeVries 
discloses providing presenting media and annotations to a user (col. 19, 11. 29-42). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to combine W3C, 
Logan and deVries as the markup language of W3C was intended to describe how media should 
interact with each other when it is displayed. 

9. Claim 21 remains rejected under 35 U.S.C 103(a) as being unpatentable over W3C 
and deVries as applied to claim 20 above, and further in view of Media Weaver, by Sha Xin 
Wei (6/2/1997). 

Regarding dependent claim 21, deVries discloses that annotations are associated with a time 
(col. 16, 11. 42-54). Both W3C and Logan are silent as to converting media. Xei discloses 
converting media (p. 12 bullet 3). It would have been obvious to one of ordinary skill in the art at 
the time of the invention modify Xei into W3C and Logan so that the annotations would still be 
presented in the same context as in the original media. 

Response to Arguments 
1 1 . Applicant's arguments filed 1/27/2004 have been fully considered but they are not 
persuasive. 

Regarding Applicant's remarks on §101 rejections: 

Applicant has alleged that the Office has ignored its argument as to the statutory nature of claims 
10-12 and 25-27. The Office submits that it was a complete response. The Office's position is 
that the claims recite non-functional descriptive data as described in the rejection above. Non- 



Application/Control Number: 09/396,984 Page 1 1 

Art Unit: 2179 

functional descriptive data is defined by the MPEP 2106.IV.B.1 as non-statutory regardless of its 
medium. If Applicant wishes to traverse the rejection the Office recommends specifically 
traversing the finding of the claims to be non- functional descriptive data. The Office has 
provided reasoning as to why the claims are non- functional descriptive data. Any legal basis that 
might be missing form the action can be found in MPEP 2 106. IV. B. 1. The burden is now on the 
Applicant to prove that the finding of non- functional descriptive data is wrong. In response to 
the specific argument set forth: 

i. Applicant alleges that §101 contemplates such claimed subject matter. The Office 
has held the claims to be directed toward non-functional descriptive data. Non- functional 
descriptive data does not comply with §101 as set forth in MPEP 2 106. IV. B. 1. 

ii. Applicant alleges a full development of reasons for rejection has not been 
provided. As noted above, the claims recite two data fields that have no function and 
merely describe data. This deemed to be ample development of reason. 

iii. Applicant alleges there is no legal authority for the rejections. 35 U.S.C. §101 is 
the legal basis for this rejection. See MPEP 2 106. IV. B. 1 for the basis for the statute not 
including non- functional descriptive data. 

iv. Applicant alleges again that there is no development of reasons as set forth in 
MPEP §706. 03(a). This as been addressed in section (ii) above. 

v. Applicant alleges that by way of being an invention employing computer readable 
media that it is statutory. Merely claiming nonfunctional descriptive material stored in a 
computer-readable medium does not make it statutory. Such a result would exalt form 
over substance. In re Sarkar, 588 F.2d 1330, 1333, 200 USPQ 132, 137 (CCPA 1978). 



Application/Control Number: 09/396,984 Page 12 

Art Unit: 2179 

vi. Applicant alleges 35 U.S.C. §112 requires that Applicant to present claims in the 
manner which it regards invention. The Office submits that that the Applicant must 
adhere to all statutes of patent law. Complying with §112 does not preclude complying 
with §101. 

vii. Applicant alleges that the rejections do not comply with MPEP §2111 .01 .H A, 
assumed to be intended to be MPEP §2106.ILA. As stated in §2106.1, "These Guidelines 
do not constitute substantive rulemaking and hence do not have the force and effect of 
law. These Guidelines have been designed to assist Office personnel in analyzing claimed 
subject matter for compliance with substantive law. Rejections will be based upon the 
substantive law and it is these rejections, which are appealable. Consequently, any 
failure by Office personnel to follow the Guidelines is neither appealable nor 
petitionable" The rejections were made in view of case law cited in the rejection and 
MPEP §2106.IV.B. 1 . The examiner is not required to follow these guidelines. 

viii. Applicant alleges that Applicant is entitled to employ function description. This 
section of the MPEP entitles the Applicant to do so without rendering the claim indefinite 
or un-enabled. The Office has made neither claim; therefore this has no bearing on the 
discussion of statutory subject matter. Again the Office points out that compliance with 
§1 12 does not alleviate Applicant of their responsibility to comply with §101. 

ix. Applicant alleges the deVries reference does not comply with §101. The Office is 
not permitted to comment on the validity of existing patents. Regardless, the 
validity/non- validity of any patent has no bearing on the current claims. 
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x. Applicant alleges that the Office has not fully responded. The Office disagrees, 
but submits that Applicant's response has now been answered. 
Regarding Applicant's remarks on claim 1, starting on p. 26: 

Applicant argues that the text tag does not qualify as annotations. The Office disagrees. The 
text is contained with a <par> tag (§ 4.2, 1), with the <switch> block. Within the switch block, as 
noticed in the example's text, are "alternative parts designed for screens with different 
resolutions, and bit-depths." The ellipses at 6, 9, and 12, are clearly meant to be the alternative 
parts. These are the different versions of multimedia content. They would be represented by one 
of the tags disclosed in §4.2.3. Applicant states "Note that no other element is provided that 
could represent any other media content." Again the Office directs the Applicant to §4.2. 1 where 
the <par> tag is explained. In the case of the example, the par tag is intended to have, at the 
ellipses, the different media information, most likely a video. The ellipses are used to show the 
flexibility. Both those tags, and <text> when contained within a <par> can be associated to 
particular time frames (p. 12). While W3C indeed does not make mention of the word 
"annotation," it is still the position of the Office, that text associated with media is an annotation. 
As the text is assigned to a particular "temporal location in the multimedia presentation," it is 
equivalent to an annotation as defined in the specification, (page 2, line 19 - page 3, line 7), 
even meeting the specific example of "temporal location " Therefore the example in question 
sets out a text element, which in addition to being media itself is text assigned to a temporal 
location with several media defined within the <switch> This is the Applicant's definition of 
annotation. The title of section that Applicant points out is referring to the elements within the 
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switch. The ellipses within the text tag do not only indicate that the source is arbitrary, but also 
that any attribute could be contained within, such as the ones listed in §4.2.3. 

Applicant also again alleges the Office has admitted in the previous responses that the 
W3C is silent with respect to annotations. That is not an accurate assessment of what is said in 
this and the previous actions. The Office merely acknowledged that the word "annotation" was 
not used. Just because the same claim language is not used, does not mean that W3C is absent of 
annotations. The Office has explained in the rejection and with the above argument why W3C 
does in fact teach annotations as defined in the specification. 

Applicant finally alleges that that W3C does not teach the entire text of every 
independent claim. The Office assumes the Applicant is merely summarizing the above 
argument, since all rejections are obviousness rejections and therefore W3C would not teach all 
elements of the claim. In either case the argument is deemed answered above. 

In regard to the remarks on the deVries patent, again the Applicant merely recites the 
claims and alleges the source does not teach the claims. This does not constitute a valid 
argument because they amount to a general allegation that the claims define a patentable 
invention without specifically pointing out how the language of the claims patentably 
distinguishes them from the references. Additionally, though deVries has not been relied on for 
teaching that an annotation is associated with multiple versions of multimedia content, Applicant 
alleges that it does not contain such a teaching. Despite the one section of deVries the Applicant 
quotes, the entire specification, exemplified by Fig. 7, teaches annotation is associated with 
multiple versions of multimedia content. The above rejections have all been rejected under 35 
U.S .C. 103 under combinations of references. In response to applicants argument that deVries 
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does not teach the entire claims, the test for obviousness is not whether the features of a 

secondary reference may be bodily incorporated into the structure of the primary reference; nor 

is it that the claimed invention must be expressly suggested in any one or all of the references. 

Rather, the test is what the combined teachings of the references would have suggested to those 

of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981), 

Regarding dependent claim Applicant's remarks regarding claim 13, 

Applicant again alleges that W3C does not teach annotations. This argument has been addressed 

in the discussion of claim 1 above. 

Regarding Applicant's remarks on claims 19 and 23: 

Specific analysis of claims 19 and 23 are now included. 

Regarding Applicant's remarks on Claims 29-34: 

As requested a new action containing rejections of claims 29-34 has been provided. 
Regarding Applicant's remarks on the MPEP: 

Applicant merely summarizes the obviousness requirements. These remarks are considered to be 
non-persuasive as Applicant is not pointing out which of these criteria the Office has not met, 
nor which parts of the rejections these are relevant to, nor how these specifically apply to the 
Applicant's claims. 

Regarding Applicant's remarks regarding claims 2-6, 8, 9, 11, 12, 14-19, 21-24, and 26-27: 

Applicant alleges these claims are allowable in view of their base claims being allowable. As the 
rejections have been maintained and no substantive arguments have been provided as to the 
dependent claims patentability, the claims remain rejected. 
Regarding Applicant's remarks on 18-19, regarding Examiner's Response: 
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Applicant again alleges the previous Examination was deficient without citing ANY specific 
examples. The Office sees no deficiencies its prior response to Applicant's previous arguments. 
The Applicant makes no allegations of what arguments in particular were deficient. Applicant 
alleges repeatedly that the Office has not addressed all arguments, and various deficiencies 
within the arguments without providing any indication of what in particular arguments are 
deficient. As such the Office can not respond to such allegations. 

Applicant alleges there was no response to the arguments with respect to anticipation as there are 
no anticipation rejections in the previous Office Action. Additionally the cited portion of the 
MPEP, specifically the portion labeled "Examiner's Note," in addition to being located nowhere 
near the previous cited portion, has been taken completely out of context. The Note relates to a 
form paragraph used by the Office, which has not been used in this or the previous rejections. 

Conclusion 

10. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, TfflS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1. 136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
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however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Adam M Queler whose telephone number is (703) 308-5213. 
The examiner can normally be reached on Monday-Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather R Herndon can be reached on (703) 308-5186. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



AQ 




STEPHEN S. HONQii 
PRIMARY EXAMINER 



